DIFFUSION MODELS

LAD: LoRA-Adapted Diffusion (2025.emnlp-demos)

Challenge:	Existing diffusion models fail to address the challenges of generating high-quality images from textual descriptions due to its large vocabulary size and complex character relationships.
Approach:	They propose a framework that integrates Chinese diffusion models with Alibaba Cloud's Platform for AI and enables the generation of contextually relevant images.
Outcome:	The proposed framework integrates with Alibaba Cloud’s Platform for AI, providing accessible and scalable solutions.

Copied to clipboard

Ruurd Jan Anthonius Kuiper, Lars de Groot, Bram van Es, Maarten van Smeden, Ayoub Bagheri

Green & Sustainable NLP

Challenge:	Autoregressive models dominate text generation but suffer from left-to-right decoding constraints that limit efficiency and bidirectional reasoning.
Approach:	They propose a framework for non-autoregressive generation that adapts LLaMA models for iterative, bidirectional sequence refinement using LoRA adapters.
Outcome:	The proposed framework adapts LLaMA models for iterative, bidirectional sequence refinement using LoRA adapters.

Text Adversarial Purification as Defense against Adversarial Attacks (2023.acl-long)

Copied to clipboard

Linyang Li, Demin Song, Xipeng Qiu

Robustness in NLP

Challenge:	Existing methods for defending against adversarial examples are difficult due to the discrete nature of texts.
Approach:	They propose a novel adversarial purification method that aims to remove adversarials and make correct predictions based on the recovered clean samples.
Outcome:	The proposed method can defend against word-substitution adversarial attacks using language models.

Benchmarking Diffusion Models for Machine Translation (2024.eacl-srw)

Copied to clipboard

Yunus Demirag, Danni Liu, Jan Niehues

Machine Translation

Challenge:	Diffusion models have shown great potential on many generative tasks, but their application to natural language processing (NLP) is still a less explored direction.
Approach:	They adapt two diffusion-based text generation models, Diffusion-LM and DiffuSeq, to perform machine translation.
Outcome:	The proposed models struggle more on long-range dependencies than other models.

Rapid Diffusion: Building Domain-Specific Text-to-Image Synthesizers with Fast Inference Speed (2023.acl-industry)

Copied to clipboard

Bingyan Liu, Weifeng Lin, Zhongjie Duan, Chengyu Wang, Wu Ziheng, Zhang Zipeng, Kui Jia, Lianwen Jin, Cen Chen, Jun Huang

Hybrid Autoregressive-Diffusion Model for Real-Time Sign Language Production (2026.acl-long)

Challenge:	Text-to-Image Synthesis (TIS) aims to generate images based on textual inputs . but, current diffusion-based models lack entity knowledge and low inference speed .
Approach:	They propose a framework for training and deploying latent diffusion models with rich entity knowledge injected and optimized networks.
Outcome:	The proposed framework improves image quality and inference speed and can be used in industrial applications.

Copied to clipboard

Maoxiao Ye, Xinfeng Ye, Sathiamoorthy Manoharan

Challenge:	Existing approaches to sign language production use autoregressive or diffusion models that generate one-by-one output tokens but suffer from exposure bias during inference.
Approach:	They propose a hybrid autoregressive-diffusion model that combines iterative refinement and sequential dependency modeling for Sign Language production.
Outcome:	The proposed model improves sign language production quality and real-time efficiency on PHOENIX14T and How2Sign.

Flow Matching for Conditional Text Generation in a Few Sampling Steps (2024.eacl-short)

Copied to clipboard

Vincent Hu, Di Wu, Yuki Asano, Pascal Mettes, Basura Fernando, Björn Ommer, Cees Snoek

DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models (2023.acl-long)

Challenge:	Current diffusion models face multiple drawbacks including slow sampling, noise schedule sensitivity, and misalignment between training and sampling stages.
Approach:	They propose a method which leverages flow matching for conditional text generation.
Outcome:	The proposed method can generate text in a few steps by training with a novel anchor loss, alleviating the need for expensive hyperparameter optimization of the noise schedule prevalent in diffusion models.

Copied to clipboard

Zijie J. Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, Duen Horng Chau

Inference-Time Feedback for Reasoning Controllability in Diffusion Language Models (2026.acl-srw)

Challenge:	Recent advances in diffusion models have enabled high-quality image generation . generating images with desired details requires proper prompts .
Approach:	They analyze syntactic and semantic characteristics of diffusion models and their prompts . they pinpoint specific hyperparameter values and prompt styles that can lead to model errors .
Outcome:	The first large-scale text-to-image prompt dataset totals 6.5TB . it contains 14 million images generated by Stable Diffusion, 1.8 million unique prompts, and hyperparameters specified by real users.

Copied to clipboard

Clovis Barbour, Huixin Zhan

EasyGen: Easing Multimodal Generation with BiDiffuser and LLMs (2024.acl-long)

Challenge:	In scientific NLP systems, model outputs often serve as interfaces to downstream systems that assume strict structural requirements.
Approach:	They evaluate machine-checkable controllability along three axes: structural control, iterative correction, and decoding dynamics.
Outcome:	The proposed model can be usefully decomposed into global structure versus local control . the proposed model improves global structure while improving iterative correction .

Copied to clipboard

Xiangyu Zhao, Bo Liu, Qijiong Liu, Guangyuan Shi, Xiao-Ming Wu

HierDiffuse: Progressive Diffusion for Robust Interest Fusion in CTR Prediction (2025.emnlp-industry)

Challenge:	Existing multimodal models that depend on encoders like CLIP or ImageBind need ample amounts of training data to bridge modalities.
Approach:	They propose an efficient model that leverages bidirectional conditional diffusion model to foster more efficient modality interactions.
Outcome:	The proposed model is able to train a projection layer linking an LLM and an adapter to align the LLM’s text space with the bidirectional diffusion model.

Copied to clipboard

Ziheng Ni, Congcong Liu, Yuying Chen, Zhiwei Fang, Changping Peng, Zhangang Lin, Ching Law, Jingping Shao

Robustness in NLP

Challenge:	Existing approaches fuse long-term behavioral profiles and short-term interactions, suffering from representational misalignment and noise in transient signals.
Approach:	They propose a framework that redefines interest fusion as a hierarchical denoising process through diffusion models.
Outcome:	The proposed framework redefines interest fusion as a hierarchical denoising process through diffusion models.

WaveFM: A High-Fidelity and Efficient Vocoder Based on Flow Matching (2025.naacl-long)

Copied to clipboard

Tianze Luo, Xingchen Miao, Wenbo Duan

AnyTrans: Translate AnyText in the Image with Large Scale Models (2024.findings-emnlp)

Challenge:	Flow matching is a robust and stable approach to training diffusion models, but it can result in subpar audio quality.
Approach:	They propose a reparameterized flow matching model for mel-spectrogram conditioned speech synthesis that uses a mel prior instead of a standard Gaussian prior to minimize unnecessary transportation costs.
Outcome:	The proposed model improves sample quality and generation speed for speech vocoders while reducing transportation costs.

Copied to clipboard

Zhipeng Qian, Pei Zhang, Baosong Yang, Kai Fan, Yiwei Ma, Derek Wong, Xiaoshuai Sun, Rongrong Ji

Multilinguality

Challenge:	Recent advances in natural language processing and computer vision have made it possible to translate images with text in one language into equivalent images displaying that text translated into another language.
Approach:	They propose an all-encompassing framework for the task–In-Image Machine Translation (IIMT) that incorporates contextual cues from both textual and visual elements during translation.
Outcome:	The proposed framework can be constructed using open-source models and requires no training, making it highly accessible and expandable.

Negative-Aware Diffusion Process for Temporal Knowledge Graph Extrapolation (2026.findings-eacl)

Copied to clipboard

Yanglei Gan, Peng He, Yuxiang Cai, Run Lin, Guanyu Zhou, Qiao Liu

TermDiffuSum: A Term-guided Diffusion Model for Extractive Summarization of Legal Documents (2025.coling-main)

Challenge:	Temporal Knowledge Graphs (TKGs) are dynamic structures representing entities and their evolving relationships through time.
Approach:	They propose a non-parametric model that encodes subject-centric histories into sequential embeddings.
Outcome:	The proposed model encodes subject-centric histories of entities, relations and temporal intervals into sequential embeddings.

Copied to clipboard

Xiangyun Dong, Wei Li, Yuquan Le, Zhangyue Jiang, Junxi Zhong, Zhong Wang

Information Extraction & Text Mining

Challenge:	Recent studies have explored diffusion models for extractive summarization task, showcasing their remarkable capabilities.
Approach:	They propose a term-guided diffusion model for extractive summarization of legal documents that incorporates legal terminology into the model via a well-designed multifactor fusion noise weighting schedule.
Outcome:	The proposed model outperforms existing models on a self-constructed legal summarization dataset and achieves improvements of 3.10, 2.84, and 2.89 on three public datasets.

Bringing Emerging Architectures to Sequence Labeling in NLP (2026.eacl-long)

Copied to clipboard

Ana Ezquerro, Carlos Gómez-Rodríguez, David Vilares

Challenge:	Pretrained Transformer encoders are the dominant approach to sequence labeling . however, few have been applied to sequence labels on flat or simplified tasks .
Approach:	They propose to use pretrained Transformer encoders to model relations across words . they find that the architectures adapt well across tagging tasks that vary in complexity .
Outcome:	The proposed architectures perform well across tagging tasks across languages and datasets.

DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models (2023.acl-long)

Copied to clipboard

Zhengfu He, Tianxiang Sun, Qiong Tang, Kuanning Wang, Xuanjing Huang, Xipeng Qiu

A Cheaper and Better Diffusion Language Model with Soft-Masked Noise (2023.emnlp-main)

Challenge:	Existing generative masked language models have a shared training objective, i.e., denoising.
Approach:	They propose a noise schedule for the forward diffusion process that controls the degree of noise added at each step based on the information of each token.
Outcome:	The proposed model improves on existing models in terms of perplexity and BLEU score.

Copied to clipboard

Jiaao Chen, Aston Zhang, Mu Li, Alex Smola, Diyi Yang

VideoEraser: Concept Erasure in Text-to-Video Diffusion Models (2025.emnlp-main)

Challenge:	Existing diffusion models have limitations in modeling discrete data, e.g., languages . we present a novel diffusion model for language modeling inspired by linguistic features in languages based on iterative denoising .
Approach:	They propose a method that iteratively denoises and adds corruptions to the textual data through soft-masking to better noise it.
Outcome:	The proposed model achieves better generation quality and lower training cost than current models with better performance.

Copied to clipboard

Naen Xu, Jinghuai Zhang, Changjiang Li, Zhi Chen, Chunyi Zhou, Qingming Li, Tianyu Du, Shouling Ji

Words Worth a Thousand Pictures: Measuring and Understanding Perceptual Variability in Text-to-Image Generation (2024.emnlp-main)

Challenge:	Experimental results show that VideoEraser outperforms prior methods regarding efficacy, integrity, fidelity, robustness, and generalizability.
Approach:	They propose a training-free framework that prevents T2V diffusion models from generating videos with undesirable concepts even when explicitly prompted with those concepts.
Outcome:	The proposed framework outperforms existing methods in erasure, celebrity erasion, and explicit content erasing tasks.

Copied to clipboard

Raphael Tang, Crystina Zhang, Lixinyu Xu, Yao Lu, Wenyan Li, Pontus Stenetorp, Jimmy Lin, Ferhan Ture

Principled Self-Correction in Discrete Diffusion: A UCB-Guided Framework for Text Generation (2026.eacl-long)

Challenge:	Current diffusion models do not cover recent models, thus we curate three test sets for evaluation.
Approach:	They propose a human-calibrated measure of variability in a set of images bootstrapped from existing image-pair perceptual distances.
Outcome:	The proposed model outperforms nine baselines by 18 points in accuracy and matches graded human judgements 78% of the time.

Copied to clipboard

Masaki Asada, Makoto Miwa

SCMAPR: Self-Correcting Multi-Agent Prompt Refinement for Complex-Scenario Text-to-Video Generation (2026.findings-acl)

Challenge:	Existing diffusion models are trained on corrupted ground-truth tokens, but at inference time they must denoise inputs corruptes from their own predictions.
Approach:	They propose a framework that denoises inputs corrupted from their own predictions at inference time.
Outcome:	The proposed framework achieves higher faithfulness and coherence over existing diffusion baselines.

Copied to clipboard

Chengyi Yang, Pengzhen Li, Jiayin Qi, Aimin Zhou, Ji Wu, Ji Liu

Discrete Diffusion Language Model for Efficient Text Summarization (2025.findings-naacl)

Challenge:	Text-to-Video (T2V) generation is a challenge under complex scenarios.
Approach:	They propose a scenario-aware and self-correcting multi-agent prompt refinement framework for T2V prompting.
Outcome:	The proposed framework improves text-to-video alignment and overall generation quality under complex scenarios.

Copied to clipboard

Do Huu Dat, Duc Anh Do, Anh Tuan Luu, Wray Buntine

LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation? (2024.naacl-long)

Challenge:	Existing discrete diffusion models fail on conditional long-text generation due to incompatibility between the backbone architectures and the random noising process.
Approach:	They propose a semantic-aware noising process that enables Transformer backbones to handle long sequences effectively.
Outcome:	The proposed model outperforms existing models on three benchmark summarization datasets while achieving much faster inference speed compared to autoregressive models.

Copied to clipboard

Yuchi Wang, Shuhuai Ren, Rundong Gao, Linli Yao, Qingyan Guo, Kaikai An, Jianhong Bai, Xu Sun

FastDiff 2: Revisiting and Incorporating GANs and Diffusion Models in High-Fidelity Speech Synthesis (2023.findings-acl)

Challenge:	Existing models for text-to-image generation have been underperforming in image-totext generation tasks.
Approach:	They propose a framework that uses a split BERT to create a dedicated latent space for captions and integrates a regularization module to manage varying text lengths.
Outcome:	The proposed framework achieves state-of-the-art performance on the MS COCO dataset with 38.2 BLEU@4 and 126.2 CIDEr .

Copied to clipboard

Rongjie Huang, Yi Ren, Ziyue Jiang, Chenye Cui, Jinglin Liu, Zhou Zhao

DiffusionDialog: A Diffusion Model for Diverse Dialog Generation with Latent Space (2024.lrec-main)

Challenge:	Experimental results show that Generative adversarial networks sacrifice sample diversity for quality and speed, while diffusion models exhibit outperformed sample quality and diversity at a high computational cost.
Approach:	They propose to combine GANs and diffusion probabilistic models to achieve better sample quality and diversity.
Outcome:	The proposed models outperform GANs and diffusion models in speech synthesis . the proposed models enjoy an efficient 4-step sampling process and exhibit better sample diversity .

Copied to clipboard

Jianxiang Xiang, Zhenhua Liu, Haodong Liu, Yin Bai, Jia Cheng, Wenliang Chen

Dialogue Systems & Conversational Agents

Challenge:	Existing studies have tried to introduce discrete or Gaussian-based latent variables to address the one-to-many problem, but the diversity is limited.
Approach:	They propose a diffusion model to enhance the diversity of dialogue generation by using continuous latent variables instead of discrete ones.
Outcome:	The proposed model greatly enhances diversity of dialog response while keeping the coherence.

Mixture-of-Subspaces in Low-Rank Adaptation (2024.emnlp-main)

Copied to clipboard

Taiqiang Wu, Jiahao Wang, Zhe Zhao, Ngai Wong

Reasoning

Challenge:	Using a subspace-inspired Low-Rank Adaptation method, large language models can be optimized for downstream tasks using parameter-efficient finetuning.
Approach:	They propose a subspace-inspired Low-Rank Adaptation method that decomposes LoRA weights into two subspaces and merges them into the frozen original weight.
Outcome:	The proposed method outperforms LoRA on commonsense reasoning, visual instruction tuning, and subject-driven text-to-image generation tasks.

DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoising (2025.acl-long)

Copied to clipboard

Zhenhao Li, Huichi Zhou, Marek Rei, Lucia Specia

Robustness in NLP

Challenge:	Existing adversarial defense methods for natural language processing still pose challenges to adversarials.
Approach:	They propose a novel adversarial defense method that incorporates a diffusion layer as a denoiser between the encoder and the classifier.
Outcome:	The proposed method improves over existing adversarial defense methods and achieves state-of-the-art performance against black-box and white-box adversarials.

Controllable Conversation Generation with Conversation Structures via Diffusion Models (2023.findings-acl)

Copied to clipboard

Jiaao Chen, Diyi Yang

Dialogue Systems & Conversational Agents

Challenge:	Current generation models fail to effectively utilize rich linguistic and world knowledge to generate coherent long text.
Approach:	They propose a conversation generation framework that incorporates human knowledge and conversation structures with both controllability and interpretability for better conversation generation.
Outcome:	The proposed framework incorporates human knowledge and conversation structures with both controllability and interpretability for better conversation generation.

Mustango: Toward Controllable Text-to-Music Generation (2024.naacl-long)

Copied to clipboard

Jan Melechovsky, Zixun Guo, Deepanway Ghosal, Navonil Majumder, Dorien Herremans, Soujanya Poria

Effective Integration of Text Diffusion and Pre-Trained Language Models with Linguistic Easy-First Schedule (2024.lrec-main)

Challenge:	Mustango is a text-to-music system that allows music-domain-knowledge-informed text-based music generation.
Approach:	They propose a music-domain-knowledge-inspired text-to-music system based on diffusion that generates music with captions that include specific instructions related to chords, beats, key and tempo.
Outcome:	The proposed system outperforms existing models in music generation tasks.

Copied to clipboard

Yimin Ou, Ping Jian

Private Synthetic Text Generation with Diffusion Models (2025.naacl-long)

Challenge:	Existing noise schedules in text diffusion models do not take into account the linguistic differences among tokens, which violates the easy-first policy for text generation.
Approach:	They propose to use a pre-trained decoder to convert denoised embedding vectors into natural language instead of the widely used rounding operation.
Outcome:	The proposed model outperforms existing models on the E2E dataset and five controllable tasks on the discrete nature of text data.

Copied to clipboard

Sebastian Ochs, Ivan Habernal

A Couch Potato is not a Potato on a Couch: Prompting Strategies, Image Generation, and Compositionality Prediction for Noun Compounds (2025.findings-acl)

Challenge:	Recent research shows diffusion models are capable of generating synthetics texts . but are they also good in generating private data if the training was under differential privacy?
Approach:	They examine whether diffusion models are capable of generating synthetics texts . they critically assess previous work on private text generation with diffusion models .
Outcome:	The proposed model outperforms auto-regressive models in generating private images despite unmet privacy assumptions . the proposed model is open-source and can be used for other purposes .

Copied to clipboard

Sinan Kurtyigit, Diego Frassinelli, Carina Silberer, Sabine Schulte Im Walde

ICG: Improving Cover Image Generation via MLLM-based Prompting and Personalized Preference Alignment (2025.emnlp-main)

Challenge:	a new method to predict the compositionality of English noun compounds is proposed .
Approach:	They propose a visual modality and vision transformers to predict the compositionality of English noun compounds.
Outcome:	The proposed method compared with a state-of-the-art text-based approach reveals complementary contributions regarding features and degrees of abstractness in English noun compounds.

Copied to clipboard

Zhipeng Bian, Jieming Zhu, Qijiong Liu, Wang Lin, Guohao Cai, Zhaocheng Du, Jiacheng Sun, Zhou Zhao, Zhenhua Dong

ProSE: Diffusion Priors for Speech Enhancement (2025.naacl-long)

Challenge:	Large language models and diffusion models have opened new possibilities for AI-generated content . personalized cover image generation remains underexplored despite its critical role in boosting user engagement on digital platforms.
Approach:	They propose a framework that integrates MLLM-based prompting with personalized preference alignment to generate high-quality, contextually relevant covers.
Outcome:	The proposed framework improves image quality, semantic fidelity, and personalization, leading to stronger user appeal and offline recommendation accuracy in downstream tasks.

Copied to clipboard

Sonal Kumar, Sreyan Ghosh, Utkarsh Tyagi, Anton Jeran Ratnarajah, Chandra Kiran Reddy Evuru, Ramani Duraiswami, Dinesh Manocha

Few-shot Temporal Pruning Accelerates Diffusion Models for Text Generation (2024.lrec-main)

Challenge:	deterministic deep learning models have been used for speech enhancement, but generative models have shown promise.
Approach:	They propose a method to apply diffusion probabilistic models to speech enhancement using priors in a latent space.
Outcome:	The proposed method achieves state-of-the-art performance on synthetic and real-world datasets while consuming less computational costs.

Copied to clipboard

Bocheng Li, Zhujin Gao, Yongxin Zhu, Kun Yin, Haoyu Cao, Deqiang Jiang, Linli Xu

Low-Resource NLP

Challenge:	Existing acceleration methods for text generation ignore the importance of the distribution of sampling steps, resulting in slow sampling rates.
Approach:	They propose a technique to accelerate diffusion models for text generation without additional training by using a Bayesian optimization approach.
Outcome:	The proposed technique achieves 400x acceleration even with minimal sampling steps after down to less than 1 minute of optimization yielding a competitive performance even with minimum sampling steps.

SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control (2023.acl-long)

Copied to clipboard

Xiaochuang Han, Sachin Kumar, Yulia Tsvetkov

FlashAudio: Rectified Flow for Fast and High-Fidelity Text-to-Audio Generation (2025.acl-long)

Challenge:	Existing diffusion models for continuous-valued domains have not been adopted for text data.
Approach:	They propose a diffusion-based language model with two key design choices . semi-autoregressive model generates blocks of text and allows local context updates . they evaluate it on unconstrained text generation benchmarks .
Outcome:	The proposed model outperforms autoregressive models on unconstrained text generation benchmarks on uncontrolled text generation.

Copied to clipboard

Huadai Liu, Jialei Wang, Rongjie Huang, Yang Liu, Heng Lu, Zhou Zhao, Wei Xue

DiffuSum: Generation Enhanced Extractive Summarization with Diffusion (2023.findings-acl)

Challenge:	Recent advances in latent diffusion models (LDMs) have markedly enhanced text-to-audio generation, yet their iterative sampling processes impose substantial computational demands, limiting practical deployment.
Approach:	They propose to learn straight flow for fast simulation by using flashAudio with rectified flows and immiscible flow to minimize the total distance of data-noise pairs in a batch vias assignment.
Outcome:	The proposed method can learn straight flow for fast simulations and reduce noise distribution.

Copied to clipboard

Haopeng Zhang, Xiao Liu, Jiawei Zhang

Information Extraction & Text Mining

Challenge:	Existing methods for extractive summarization are formulated as a sequence labeling problem by making individual sentence label predictions.
Approach:	They propose a novel paradigm for extractive summarization by directly generating summary sentences with diffusion models and extracting sentences based on sentence representation matching.
Outcome:	The proposed framework achieves state-of-the-art extractive results on CNN/DailyMail with ROUGE scores of 44.83/22.56/40.56.

Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data? (2025.findings-acl)

Copied to clipboard

Che Liu, Zhongwei Wan, Haozhe Wang, Yinda Chen, Talha Qaiser, Chen Jin, Nikolay Burlutskiy, Fariba Yousefi, Rossella Arcucci

InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation (2023.findings-emnlp)

Challenge:	Medical Vision-Language Pretraining (MedVLP) models typically require large-scale datasets with paired, high-quality image-text data.
Approach:	They propose to generate large-scale synthetic image-text pairs using off-the-shelf generative models . they propose to isolate model and training settings, focusing entirely from the data perspective.
Outcome:	The proposed pipeline outperforms models trained on real data by 3.8% on averaged AUC on zero-shot classification tasks.

Copied to clipboard

Renzhi Wang, Jing Li, Piji Li

Do It Yourself (DIY): Modifying Images for Poems in a Zero-Shot Setting Using Weighted Prompt Manipulation (2025.emnlp-main)

Challenge:	Existing text diffusion models have failed to capture the difference between the “easy-first” process and the “keyword-first" process of humans.
Approach:	They propose a non-autoregressive text diffusion model that incorporates a "keyinfo-first" generation strategy and a noise schedule based on the amount of text information.
Outcome:	The proposed model outperforms the baseline model in terms of generation quality and diversity, and higher sampling efficiency.

Copied to clipboard

Sofia Jamil, Kotla Sai Charan, Sriparna Saha, Koustava Goswami, Joseph K J

GOME: Grounding-based Metaphor Binding With Conceptual Elaboration For Figurative Language Illustration (2024.emnlp-main)

Challenge:	a novel method to enhance imagery in poetic language is proposed . weighted prompt manipulation is a new approach to enhance poetry images . current diffusion models struggle to interpret metaphorical language, symbolism, and nuanced themes.
Approach:	They propose a weighted prompt manipulation technique that modifies attention weights and text embeddings within diffusion models to enhance or suppress specific words' influence in the final generated image.
Outcome:	The proposed technique enhances or suppresses the influence of specific words in the final generated image, leading to semantically richer and more contextually accurate visualizations.

Copied to clipboard

Linhao Zhang, Jintao Liu, Li Jin, Hao Wang, Kaiwen Wei, Guangluan Xu

Visual Data in NLP

Challenge:	Existing Large Language Models (LLMs) and multimodal models are unable to illustrate figurative language based on literal objects, ignoring the underlying groundings and associations across disparate metaphorical domains.
Approach:	They propose a grounding-based method for metaphor illustration that integrates metaphorical knowledge into systematic instructions for existing large language models.
Outcome:	The proposed method is superior to existing LLMs, diffusion models, or their direct collaboration.

TESS 2: A Large-Scale Generalist Diffusion Language Model (2025.acl-long)

Copied to clipboard

Jaesung Tae, Hamish Ivison, Sachin Kumar, Arman Cohan

STRICT: Stress-Test of Rendering Image Containing Text (2025.emnlp-main)

Challenge:	Existing instruction-following diffusion models are predominantly trained using an autoregressive paradigm.
Approach:	They propose a general instruction-following diffusion language model that outperforms contemporary instruction-tuned diffusion models and matches and sometimes exceeds strong autoregressive (AR) models.
Outcome:	The proposed model outperforms and sometimes exceeds existing autoregressive (AR) models on a number of tasks.

Copied to clipboard

Tianyu Zhang, Xinyu Wang, Lu Li, Zhenghan Tai, Jijun Chi, Jingrui Tian, Hailin He, Suyuchen Wang

Automatic Evaluation for Text-to-image Generation: Task-decomposed Framework, Distilled Training, and Meta-evaluation Benchmark (2025.acl-long)

Challenge:	Despite the advances in diffusion models, the generation of coherent text remains a major bottleneck.
Approach:	They propose a benchmark to test the ability of diffusion models to render coherent text in images.
Outcome:	The proposed model fails to generate coherent and legible text in images despite its iterative nature . the model fails in both the maximum length of readable text and correctness and legibility of the generated text .

Copied to clipboard

Rong-Cheng Tu, Zi-Ao Ma, Tian Lan, Yuehao Zhao, Heyan Huang, Xian-Ling Mao

DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak (2025.emnlp-main)

Challenge:	Existing MLLMs rely on commercial models such as GPT-4o for evaluations, but they are not universally accessible.
Approach:	They propose a task decomposition evaluation framework based on GPT-4o to automatically construct a specialized training dataset to break down the multifaceted evaluation process into simpler sub-tasks.
Outcome:	The proposed framework outperforms the current state-of-the-art GPT-4o evaluation framework with over 4.6% improvement in Spearman and Kendall correlations with human judgments.

Copied to clipboard

Hao Wang, Hao Li, Junda Zhu, Xinyuan Wang, Chengwei Pan, Minlie Huang, Lei Sha

Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning (2026.acl-long)

Challenge:	Large Language Models (LLMs) are susceptible to generating harmful content when prompted with carefully crafted inputs, a vulnerability known as LLM jailbreaking.
Approach:	They propose an end-to-end generative approach for jailbreak rewriting inspired by diffusion models that uses a sequence-tosequence (seq2sequ) diffusion model as a generator, conditioning on the original prompt and guiding the denoising process with a novel attack loss.
Outcome:	Experiments on Advbench and Harmbench show that the proposed method outperforms autoregressive jailbreak models across evaluation metrics including ASR, fluency, diversity and diversity.

Copied to clipboard

Chenyang Shao, Sijian Ren, Fengli Xu, Yong Li

Mechanistic Interpretability of Text-to-Image Diffusion Models via Cross-Attention Interventions (2026.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated strong capabilities in complex reasoning tasks, yet their autoregressive generation paradigm makes it computationally prohibitive to explore diverse reasoning paths.
Approach:	They propose a framework that combines diffusion-based generation with autoregressive evaluation to efficiently generate diverse intermediate reasoning thoughts and employ LLMs as evaluators to assess and select candidates based on their plausibility and correctness.
Outcome:	The proposed framework improves inference efficiency while maintaining competitive or superior reasoning accuracy.

Copied to clipboard

Maisha Maliha, Dean F. Hougen

FairGen: Controlling Sensitive Attributes for Fair Generations in Diffusion Models via Adaptive Latent Guidance (2025.emnlp-main)

Challenge:	Text-to-image diffusion models generate high quality images through iterative denoising, but their internal mechanisms for grounding prompt semantics into visual structure remain unclear.
Approach:	They propose a mechanistic interpretability framework that probes how individual prompt tokens are represented and utilized during the denoising process.
Outcome:	The proposed framework enables module-wise and head-wise attribution of semantic changes across denoising timesteps.

Copied to clipboard

Mintong Kang, Vinayshekhar Bannihatti Kumar, Shamik Roy, Abhishek Kumar, Sopan Khosla, Balakrishnan Murali Narayanaswamy, Rashmi Gangadharaiah

Language Model Based Text-to-Audio Generation: Anti-Causally Aligned Collaborative Residual Transformers (2025.emnlp-main)

Challenge:	Text-to-image diffusion models often exhibit generation biases toward specific demographic groups, raising ethical concerns and limiting their adoption.
Approach:	They propose an adaptive latent guidance mechanism which controls the generation distribution during inference by dynamically adjusting the diffusion process to enforce specific attributes.
Outcome:	The proposed model outperforms existing models on HBE and Stable Bias datasets and achieves substantial bias reduction.

Copied to clipboard

Juncheng Wang, Chao Xu, Cheng Yu, Zhe Hu, Haoyu Xie, Guoqi Yu, Lei Shang, Shujun Wang

ControlText: Unlocking Controllable Fonts in Multilingual Text Rendering without Font Annotations (2025.findings-emnlp)

Challenge:	Autoregressive language models excel in text-to-audio generation, but lag behind diffusion models by a non-trivial margin.
Approach:	They propose a framework that integrates multiple isolated transformers with causal conditioning and anti-causal alignment via reinforcement learning.
Outcome:	The proposed framework outperforms existing LM-based and diffusion-based systems in audio synthesis.

Copied to clipboard

Bowen Jiang, Yuan Yuan, Xinyi Bai, Zhuoqun Hao, Alyson Yin, Yaojie Hu, Wenyu Liao, Lyle Ungar, Camillo Jose Taylor

Visual Data in NLP

Challenge:	a new method for visual text rendering requires glyph annotations to be obtained .
Approach:	They propose a model that integrates diffusion with a text segmentation model to achieve multilingual text rendering using just raw images without font label annotations.
Outcome:	The proposed model can achieve font-controllable multilingual text rendering without label annotations.

CLIPErase: Efficient Unlearning of Visual-Textual Associations in CLIP (2025.acl-long)

Copied to clipboard

Tianyu Yang, Lisen Dai, Xiangqi Wang, Minhao Cheng, Yapeng Tian, Xiangliang Zhang

Diffusion Models Through a Global Lens: Are They Culturally Inclusive? (2025.acl-long)

Challenge:	MU has gained significant attention as a means to remove the influence of specific data from a trained model without requiring full retraining.
Approach:	They propose a novel approach that disentangles and selectively forgets both visual and textual associations, ensuring that unlearning does not compromise model performance.
Outcome:	Experiments on CIFAR-100, Flickr30K, and Conceptual 12M show that CLIPErase effectively removes designated associations from multimodal samples in downstream tasks while preserving model performance on retain set.

Copied to clipboard

Zahra Bayramli, Ayhan Suleymanzade, Na Min An, Huzama Ahmad, Eunsu Kim, Junyeong Park, James Thorne, Alice Oh

Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinement (2026.findings-acl)

Challenge:	Text-to-image diffusion models have produced compelling, detailed images from text prompts, but their ability to accurately represent cultural nuances remains an open question.
Approach:	They propose a benchmark to evaluate whether diffusion models can generate culturally specific images spanning ten countries.
Outcome:	The proposed model fails to generate culturally specific images spanning ten countries . it shows significant disparities in cultural relevance, description fidelity, and realism compared to real-world reference images.

Copied to clipboard

Daeun Lee, Jaehong Yoon, Jaemin Cho, Mohit Bansal